DOCUMENT RESUME 



ED 286 940 



TM 870 607 



TITLE 

INSTITUTION 

SPOH? AGENCY 
PUB DATE 
CONTRACT 
NOTE 

AVAILABLE FROM 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Computerized Adaptive Testing. ERIC Digest. 
ERIC Clearinghouse on Tests, Measurement, and 
Evaluation, Princeton, N.J. 

National In&t. o£ Education (ED), Washington, DC. 
[83] 

400-83-0015 

4p.; ERIC/TME Update Series. 
ERIC Clearinghouse on Tests, Measurement, and 
Evaluation, Educational Testing Service, Princeton, 
NJ 08541-0001 (free). 

Information Analyses - ERIC Information Analysis 
Products (071) 

MFOl/PCOl Plus Postage. 

^Adaptive Testing; ^Computer Assisted Testing; *Item 
Banks; ^Latent Trait Theory; Testing Problems; Test 
Items 

ERIC Digests 



ABSTRACT 

This brief overview notes that an adaptive test 
differs from standardized achievement tests in that it does not 
consist of a certain set of items that are administered to a group of 
examinees. Instead, the test is individualized for each examinee. The 
items administered to the examinee are selected from a large pool of 
items on the basis of the correctness of the examinee *s previous 
responses. Several steps are involved in constructing a computerized 
adaptive test (CAT): (1) the item pool must be developed; (2) some 
item characteristics must be calculated for each item; and (3) a 
.'Strategy for deleting items from the item pool must be chosen, along 
with a scoring procedure. Latent trait theory assumes that an 
underlying trait is the sole determinant of an examinee* s responses 
to test items, and that performance on an item is governed by two 
factors: the examinee's ability and the item's difficulty. Several 
item selection strategies have been developed, including the Robbins 
Monro procedure, fixed step size, the flexilevel method, the Bayesian 
approach, and the stratif ied-adaptive approach. Advantages of CAT are 
its increased efficiency, optimization of the difficulty level of the 
test for each examinee, and administration convenience. The most 
obvious constraint on the use of CAT is the problem of availability 
of appropriate computer hardware and software. (LMO) 
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COMPUTERIZED 



An adtpcive tett it very different from 
sttndtrdized tchievenent tettt* It doet not contitt 
of t cerrftin «ct of iteat thtt tre tdminittered to t 
group of exMineet* Instead, the tett it 
individutlized for etrh extminee* The itent 
tdminittered to the extminee tre telected froa t 
Itrge pool of itemt on the bttit of the correctnest 
of the extainee*t previout retpontet. To begin the 
procett, tn inititl ettimtte of tn extainee't 
tbility level it obttined from previout Cett tcoret, 
grtde level, or tone other indicttor* Thin 
ettimtted tbility level it uted to choose tn iten 
with tn tppropritte difficulty level from t pool of 
itemt. If the item is tnsvered correctly, the next 
item telected it more difficult; if the item it 
tnsvered incorrectly, the next item telected it 
etsier* This procett it continued until the 
extminee *t true tbility level it ettimtted with 
tufficient tccurtcy* 

There tre tevertl ttept in conttructing t 
computerized adtptive tett (CAT). Firtt , the item 
pool must be developed* It mutt conttin mtny items 
tt vtrying cHfficulty levelt, til metsuring the ttae 
tbility. Ne^t, tome item chtrtcterittict , ctlled 
item ptrtmetert, must be ctlcultted for etch item. 
Then t ttrttegy for telecting itemt from the item 
pool must be chosen, tlong with t tcoring procedure. 
The choice of item selection ttrttegy tnd encoring 
procedure is made by comptring the tmount of 
information each aelection atrategy/acoring 
procedure combination providea about the examinee's 
ability. 

CAT AND UTENT TRAIT THEORY 

Latent trait theory assumes that an underlying 
trait ia the sole determinant of an examinee* a 
reaponsea to teat items; and that level of 
per.^ormance on an item ia governed by two factora: 
the examinee *a ability and the difficulty of the 
item. Mor.; uj^^uif ically , it ia assumed that for any 
given iteu, a peraon with a higher ability ahould 

x-*^ a higher chance of passing an item than a 
person with a lower ability; and for peraona with 
the aame level of ability, the chance of paasing an 
eaaier item ahould be higher than the chance of 
paaaing a more difficult item, llieae asaumptiona 
can be expreaaed mathematically. 
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ADAPTIVE TESTING 



Latent trait theory haa certain advantagea over 
claasical teat theory when applied to CAT^ Firat, 
latent trait theory allowa all the teaC acores to be 
expreaaed on one acale, regardless of the teat 
adminiatered. This allowa the easy comparison of 
examineea. Second, in the mathematical models 
developed as expressions of latent trait theory, the 
teat item parametera can be calculated independent 
of the group taking the teat, rather than being tied 
to a certain aample of examineea. 

Several different mathematical models have been 
developed to expreaa the assumptions of latent trait 
theory. These latent trait models take into account 
different item parametera. All the models use 
difficulty level as an item parameter. In addition, 
some models take into account the discriminating 
power of an item and the guessing behavior of the 
examineea. 

ITEM SELECTION STRATEGIES 

Several item aelection atrategies have been 
developed. In each of them, a more difficult item 
la selected if che examinee responds correctly; an 
easier item is selected if the examinee responds 
incorrectly. Three of the atrategies differ in the 
size of the difference in difficulty level between 
the laat item aelected and the next item selected. 
These three atrategiea are: 

Robbing Monro Procedure . The amount of 
difference in item difficulty beCween the 
current item and the next item gradually 
decreases, so that the item difficulty 
converges on the level that is equal to the 
examinee*a ability level. 

Fixed Step Size . The amount of difference in 
item difficulty is kept constant. The item 
difficulty levela do not converge; they 
vaacillate between just above and just below 
the examinee's ability level. 

Flexilevel . Thia method uses a smaller item 
pool than the previous two methods. After an 
item is anawered correctly, the next harder 
item is adminiatered. After the item which is 
equal to the examinee'a true abilitv level, the 
aelected itema will oacillate between those 
that are mi^ch eaaier and those that are much 
harder. 
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Tyo nore scrtcegies tre «lso cotoaonly used* 
With Che Bayesian tpprotch , Che exaoiaee*t abilicy 
level is esCiaaCed from each response* Icems are 
selecced such Chac che difference becveen icem 
difficulcy and che escimjted abilicy is Che minimum 
possible* In Che scracif ied-adapcive , or 
scradapcive, approach, icems are arranged inco 
scraca by difficulcy, Chen vichin a scracum by 
discriminacing power* iCem seleccion moves from 
scratum Co sCracum, scarcing vich che mosc 
discriminacing icem in each level* 

ADVANTAGES OF CAT 

CoopuCerized adapcive Cescing has several 
advanCages over group-*-adminiscered sCandardized 
Cescs* A major advancage of CAT is ics increased 
efficiency* An accurace escimace of an examinee's 
abilicy can be obcained by adminisCering fever 
icems* This is advancageous boch for che examiner 
and che examinee* 

Second, an adapcive CesC can individually 
opcimize Che difficulcy level of Che Cesc for each 
examinee* Ihis allows che CesC Co do a beccer job 
of measuring Che abilicy levels of individuals, 
especially ac che extremes of abilicy; and ic allows 
Che examinee Co experience much less fruscracion 
from responding Co icems which are much Coo 
difficulc or Coo easy* 

The CesC adminiscracion seccin^ for CAT also 
provides several advanCages* The Cesc can be 
scheduled ac che examinee's convenience* Tesc 
securicy is increased because no CesC booklecs are 
princed and each examinee's "cesc" is differenc* 
The adminiscracion of che CesC is more sCandardized 
because all adminiscracion is handled by che 
compuCer* The CesC resulcs can be made immediacely 
available Co che examinee* Finally, addicional 
informacion abouC Che examinee can be collecCed, 
such as che amounc of cime caken Co respond Co 
specific icems or Che consiscency of responses 
across icems* 

CONSTRAINTS TO USING CAT 

The mosc obvious conscrainc on che use of CAT 
is che availabilicy of appropriace compuCer hardware 
and sofcware* AdequACe sCorage space for large icem 
pools and icem paramecer daca muse be available, 
along wich programs chac will respond quickly enough 
Co keep examinee waicin^ cime ac an accepCable 
level* 



I^e icem pool icself muse be developed and muse 
be checked Co make sure ChaC all the icems in che 
icem pool are measuring che same ability. If Chey 
do noC, lacenc Craic theory is noc appropriace, and 
Che Cesc will noc provide an accurace escimaCe of 
Che examinee's abilicy. 

Finally, ic has been found ChaC, using an icem 
pool of 500 icems, you acCually gec betcer accuracy 
by adminisCering che 60 mosc discriminacing icems, 
racher Chan using adapcive Cescing Cechniques Co 
selecc che icem ac che most appropriace difficulcy 
level* This disadvantage muse be overcome Co make 
CAT Cruly praccical* 
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